modules/zstd: Add initial ZSTD Encoder procs#1846
Draft
lpawelcz wants to merge 159 commits into
Draft
Conversation
rw1nkler
force-pushed
the
zstd_encoder
branch
2 times, most recently
from
6899691 to
20c221b
Compare
rw1nkler
force-pushed
the
zstd_encoder
branch
4 times, most recently
from
77d130c to
6f9a1a0
Compare
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Add cocotb testing utilities: - XLSStruct for easier handling and serializing/deserializing XLS structs - XLSChannel that serves as a dummy receiving channel - XLSMonitor that monitors transactions on an XLS channel - XLSDriver that can send data on an XLS channel - LatencyScoreboard that can measure latency between corresponding transactions on input and output buses - File-backed AXI memory python model Add cocotb tests for: - modules/zstd/memory/MemReader - modules/zstd/memory/AxiWriter - modules/zstd/memory/MemWriter Co-authred-by: Pawel Czarnecki <pczarnecki@antmicro.com> Co-authred-by: Robert Winkler <rwinkler@antmicro.com> Co-authred-by: Michal Czyz <mczyz@antmicro.com> Signed-off-by: Michal Czyz <mczyz@antmicro.com> Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com> Signed-off-by: Robert Winkler <rwinkler@antmicro.com> Signed-off-by: Krzysztof Obłonczek <koblonczek@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Co-authored-by: Robert Winkler <rwinkler@antmicro.com> Signed-off-by: Robert Winkler <rwinkler@antmicro.com> Signed-off-by: Krzysztof Obłonczek <koblonczek@antmicro.com>
ZstdDecoder:
* Cocotb tests:
* Move third-party verilog modules (AXI Interconnect) to external directory
* Replace AXI Interconnect with AXI Crossbar that handles simultaneous AXI Read
and Write transactions
* Add reference memory and fill it with expected data for comparison
against testbench memory at the end of the decoding
Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Adjust cocotb test to: * Removal of Repacketizer proc * Removal of stream-based output channels from * SequenceExecutor * ZstdDecoder Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
* Decode multiple ZSTD frames in a single cocotb testbench * Add one cocotb testbench per type of the ZSTD frames: * Frames with RAW blocks only * Frames with RLE blocks only * Frames with Compressed blocks only (disabled) * Frames with mixed blocks (disabled) Internal-tag: [#67272] Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Caused by timeouts after rebase - looks like regression in the performance of codegen Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
…sources Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Improve formatting, wording and fix lint issues Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com> Signed-off-by: Wojciech Sipak <wsipak@antmicro.com>
Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com> Co-authored-by: Krzysztof Oblonczek <koblonczek@antmicro.com> Co-authored-by: Wojciech Sipak <wsipak@antmicro.com> Co-authored-by: Dominik Lau <dlau@antmicro.com> Co-authored-by: Szymon Gizler <sgizler@antmicro.com> Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
…er test Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
…encoder Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Co-authored by: Dominik Lau <dlau@antmicro.com>
rw1nkler
force-pushed
the
zstd_encoder
branch
2 times, most recently
from
6bd74f2 to
701e0ca
Compare
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds initial procs of the ZSTD Encoder. It's based on #1654.
Changes in this PR consist of:
AlignedParallelRamproc - Responsible for handling Read/Write operations on multiple RAM models in parallel.HashTableproc - Generic Hash Table implementation. It uses underlying RAM models for operation and is controlled with a Read/Write Request/Response channels interface. In the ZSTD Encoder, it is used for fast lookup of repeating symbols in the input data buffer.HistoryBufferproc - Data buffer capable of storing historical data and outputting said data from arbitrary chosen offset up to the top of the buffer. UsesAlignedParallelRamproc underneath.MatchFinderproc - Processes data stream from the input buffer in the system memory. Searches for repeating symbol patterns in theHashTableproc. Symbols that are not found are stored directly as literals in the literals buffer, while those that have matches are stored as sequences in the sequence buffer, using the matched offset and length information. Sequence and Literals Buffers are implemented in the system memory accessible via the AXI interface.This PR is currently a
WIP.The following work needs to be finished to mark this PR as
ready for review:MatchFindercodegen